Mouth Removal Method For Red-Eye Detection And Correction

ABSTRACT

An input image (e.g. a digital RGB color image) is subjected to an eye classifier that is targeted at discriminating a complete eye pattern from any non-eye patterns. The red-eye candidate list with associated bounding boxes that are generated by the red-eye classifier are received. The bounding rectangles are subjected to object segmentation. A connected component labeling procedure is then applied to obtain one or more red regions. The largest red region is then chosen for feature extraction. A number of features are then extracted from this region. Then these features are used to determine if the particular candidate red-eye object is a mouth.

BACKGROUND Field of Invention

Red-eye detection and correction technologies are used in printers,digital cameras, and image editing software to localize and correct thered-eye effects in digital photographs captured using a flash. Thoughthere has been a great deal of progress in red-eye detection andcorrection in the last few years, many problems remain unsolved.Research in these areas must confront many challenging problems,especially when dealing with varying illumination, low image quality andresolution, eye size and variations in face orientation, and backgroundchanges in complex scenes.

In general, early stages of a red-eye detection involve distinguishingbetween true red-eye objects and a number of incorrectly detectednon-red-eye objects (referred to as false positives or falses), whichare particularly evident in complex scenes. This false detectionprocessing can be reduced based on the evaluation of the object's color,structural and geometric characteristics, as disclosed in commonlyassigned U.S. patent application Ser. No. 12/349,911, filed Jan. 7,2009, and entitled “Method of detecting red-eye objects in digitalimages using color, structural, and geometric characteristics,” whichapplication is hereby expressly incorporated herein by reference in itsentirety. False detection processing can also be reduced based onluminance-chrominance characteristics, contrast characteristics, regionsmoothness characteristics, binary pattern characteristics and glassesframe characteristics, as disclosed in commonly assigned U.S. patentapplication Ser. No. 12/575,321, filed Oct. 7, 2009, and entitled“Automatic red-eye object classification in digital photographicimages,” which application is hereby expressly incorporated herein byreference in its entirety. False detection processing can be reducedfurther based on some trained classifier such as boosting-basedframework, as disclosed in commonly assigned U.S. patent applicationSer. No. 12/575,298, filed Oct. 7, 2009, and entitled “Automatic red-eyeobject classification in digital images using a boosting-basedframework,” which application is hereby expressly incorporated herein byreference in its entirety.

Although these techniques can significantly suppress false detections,some false positives still remain, in particular mouth patterns. This isdue to the fact that mouth patterns usually exhibit similar color andstructural characteristics as that of true red-eye objects. Correctionof the mouth area as a falsely detected red-eye object will ruin thepicture and disappoint the end user. Therefore, suppressing mouth (falsered-eye) detections becomes a critical factor in ensuring a successfulapplication in real world products.

SUMMARY OF INVENTION

An object of the present invention is to remove mouth detections (falsepositives) as red-eye candidates while preserving high-computationalspeed and eye detection rate.

The present invention assumes that an input image (e.g. a digital RGBcolor image) has been subjected to an eye classifier that is targeted atdiscriminating a complete eye pattern from any non-eye patterns. Thepresent invention starts with the red-eye candidate list with associatedbounding boxes that are generated by the red-eye classifier.

The bounding rectangles are subjected to object segmentation. As isknown, segmentation, generally, is the process of grouping together intoa single entity (object) pixels that have something in common. In thepresent invention, the purpose of object segmentation is to group orsegment out red regions (mouth or red retina).

A connected component labeling procedure is then applied to obtain oneor more red regions. The largest red region denoted as “newFeaObj” isthen chosen for feature extraction.

A number of features are then extracted from this region. These include:feature k_(w), defined as the ratio between a width of the segmented redregion and a width of the bounding rectangle; feature k_(h), defined asthe ratio between a height of the segmented red region and a height ofthe bounding rectangle; feature s, defined as the ratio between thewidth and height of the segmented red region; feature v, defined as adecision score from the eye classifier; and feature m, defined as ageometrical relationship between the particular candidate red-eye objectand others of the plurality of candidate red-eye objects.

The present invention then uses these features k_(w), k_(h), s, v, and mto determine if the particular candidate red-eye object is a mouth.

Other objects and attainments together with a fuller understanding ofthe invention will become apparent and appreciated by referring to thefollowing description and claims taken in conjunction with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings wherein like reference symbols refer to like parts:

FIG. 1 is a flowchart of the general method of the present invention;

FIG. 2 is a general block diagram of an image capture device and systemfor utilizing the present invention;

FIG. 3 is an example of a captured image in RGB color space and thebounding boxes output by a typical red-eye classifier;

FIG. 4 shows the image of FIG. 3 in normalized LAB color space;

FIG. 5 illustrates the results of segmentation of the image of FIG. 3 inLAB color space;

FIG. 6 illustrates the bounding box of eye candidates output by atypical red-eye classifier, with (a) showing a whole eye, and (b)showing a partial eye;

FIG. 7 illustrates the bounding box of mouth pattern candidates outputby a typical red-eye classifier, with (a) showing a whole mouth, and (b)showing a partial mouth;

FIG. 8 illustrates the geometrical relationship between objects in animage with (a), (b), and (c) showing three different resultant labelsfor feature m used in the present invention;

DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description, reference is made to theaccompanying drawings that show, by way of illustration, exampleembodiments of the invention. In the drawings, like numerals describesubstantially similar components throughout the several views. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention. Other embodiments may be utilizedand structural, logical and electrical changes may be made withoutdeparting from the scope of the present invention. Moreover, it is to beunderstood that the various embodiments of the invention, althoughdifferent, are not necessarily mutually exclusive. For example, aparticular feature, structure, or characteristic described in oneembodiment may be included within other embodiments. The followingdetailed description is, therefore, not to be taken in a limiting sense,and the scope of the present invention is defined only by the appendedclaims, along with the full scope of equivalents to which such claimsare entitled.

In general, example embodiments relate to methods, devices, andcomputer-readable media for detecting red-eye objects in images. Exampleembodiments can be used in conjunction with red-eye correctionapplications to produce images in which red-eye objects are detected andmodified to remove or minimize the red-eye effect. Consequently, imagequality may be enhanced automatically with little or no usermanipulation of the image.

Example embodiments detect a red-eye effect in images by evaluatingwhether or not a red-eye candidate object is a mouth or not. Methodsconsistent with the invention may be implemented in image capturedevices such as scanners or digital cameras, as well as in softwaremodules including printer drivers or image editing software, among otherthings.

With reference now to FIG. 1, an example method 100 for red-eyedetection is disclosed. More particularly, the example method 100identifies objects in the image and determines whether the objects arecandidate red-eye objects. Each candidate red-eye object may then beeliminated or retained as a candidate by deciding whether or not thecandidate object is a mouth pattern or not.

The example method 100 and variations thereof disclosed herein can beimplemented using computer-readable media for carrying or havingcomputer-executable instructions or data structures stored thereon. Suchcomputer-readable media can be any available media that can be accessedby a processor of a general purpose or special purpose computer. By wayof example, and not limitation, such computer-readable media cancomprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othermedium which can be used to carry or store program code in the form ofcomputer-executable instructions or data structures and which can beaccessed by a processor of a general purpose or special purposecomputer. Combinations of the above should also be included within thescope of computer-readable media.

Computer-executable instructions comprise, for example, instructions anddata which cause a processor of a general purpose computer or a specialpurpose computer to perform a certain function or group of functions.Although the subject matter is described herein in language specific tomethodological acts, it is to be understood that the subject matterdefined in the appended claims is not necessarily limited to thespecific acts described herein. Rather, the specific acts describedherein are disclosed as example forms of implementing the claims.

Examples of special purpose computers include image processing devicessuch as digital cameras (an example of which includes, but is notlimited to, the Epson R-D1 digital camera manufactured by Seiko EpsonCorporation headquartered in Owa, Suwa, Nagano, Japan), digitalcamcorders, projectors, printers, scanners, copiers, portable photoviewers (examples of which include, but are not limited to, the EpsonP-3000 or P-5000 portable photo viewers manufactured by Seiko EpsonCorporation), or portable movie players, or some combination thereof,such as a printer/scanner/copier combination (examples of which include,but are not limited to, the Epson Stylus Photo RX580, RX595, or RX680,the Epson Stylus CX4400, CX7400, CX8400, or CX9400Fax, and the EpsonAcuLaser® CX11NF manufactured by Seiko Epson Corporation) or aprinter/scanner combination (examples of which include, but are notlimited to, the Epson TM-J9000, TM-J9100, TM-J7000, TM-J7100, andTM-H6000III, all manufactured by Seiko Epson Corporation) or a digitalcamera/camcorder combination. An image processing device may include ared-eye detection capability, for example, to detect red-eye effects inan image. For example, an image capture device, such as a camera orscanner, with this red-eye detection capability may include one or morecomputer-readable media that implement the example method 100.Alternatively, a computer connected to the image capture device mayinclude one or more computer-readable media that implement the examplemethod 100.

A schematic representation of an example image capture device 200 isdisclosed in FIG. 2. The example image capture device 200 exchanges datawith a host computer 250 by way of an intervening interface 202.Application programs and an image capture device driver may also bestored for access on the host computer 250. When an image retrievecommand is received from the application program, for example, the imagecapture device driver controls conversion of the command data to aformat suitable for the image capture device 200 and sends the convertedcommand data to the image capture device 200. The driver also receivesand interprets various signals and data from the image capture device200, and provides necessary information to the user by way of the hostcomputer 250.

When data is sent by the host computer 250, the interface 202 receivesthe data and stores it in a receive buffer forming part of a RAM 204.The RAM 204 can be divided into a number of sections, for examplethrough addressing, and allocated as different buffers, such as areceive buffer or a send buffer. Data, such as digital image data, canalso be obtained by the image capture device 200 from the capturemechanism(s) 212, the flash EEPROM 210, or the ROM 208. For example, thecapture mechanism(s) 212 can generate a digital photographic image. Thisdigital image can then be stored in the receive buffer or the sendbuffer of the RAM 204.

A processor 206 uses computer-executable instructions stored on a ROM208 or on a flash EEPROM 210, for example, to perform a certain functionor group of functions, such as the method 100 for example. Where thedata in the receive buffer of the RAM 204 is a digital image, forexample, the processor 206 can implement the methodological acts of themethod 100 on the digital image to detect red-eye objects in the digitalimage and thereby remove or minimize red-eye effects. Further processingin an imaging pipeline may then be performed on the digital image beforethe image is displayed by the image capture device 200 on a display 214,such as an LCD display for example, or transferred to the host computer250, for example.

The example method 100 for detecting red-eye effects in an image willnow be discussed in connection with FIG. 1. Prior to performing method100, an input image can be targeted for various image processingoperations including red-eye detection. The targeted input image may bea digital color image or a digitized or scanned version of a colorimage. Various image processing techniques may be applied to thetargeted input image before method 100 is performed.

The present invention assumes that an input image (e.g. a digital RGBcolor image) has been subjected to an eye classifier that is targeted atdiscriminating a complete eye pattern from any non-eye patterns. Anexample of such classifier is the boosting based classifier disclosed incommonly assigned U.S. patent application Ser. No. 12/575,298, filedOct. 7, 2009, and entitled “Automatic red-eye object classification indigital images using a boosting-based framework,” which application ishereby expressly incorporated herein by reference in its entirety. Thepresent invention is not limited to or concerned with any particular eyeclassifier and any known classifier such as those based on an AdaBoostalgorithm may be used to generate the red eye candidate list and theirassociated bounding boxes. Other examples include, but are not limitedto, support vector machines, neural networks and other training ornon-training based classifiers. The output of such classifier is usuallya rectangular bounding box which is intended to include a complete eyeregion, but may indeed include a bounding box of a mouth region.

The present invention starts (START, step 102) with the red-eyecandidate list with associated bounding boxes that are generated by thered-eye classifier. FIG. 3 illustrates a typical example of an inputimage that has been subjected to a red-eye classifier with the resultbeing 3 candidate objects (two true red-eye objects and one mouth, falsepositive, object) being output from the classifier. The rectangles shownin FIG. 3 represent the bounding boxes output by the red-eye classifier.Based on the observation that in the detected region (bounded by therectangular bounding box), the true red-eye pattern usually containsonly a relatively few red pixels due to the fact the eye ball regionoccupies a small part of a complete eye. However, for a mouth pattern,the red area usually occupies a relatively large part of the boundingbox. Therefore, the ratio between the bounding box size and red partsize can be an effective indicator to help discriminate between an eyepattern and a mouth pattern. This distinction between true red-eyecandidates and false mouth pattern candidates is used in the presentinvention to exclude mouth patterns.

In the present invention, the bounding rectangles, for example thoseshown in FIG. 3, are subjected to object segmentation (FIG. 1, step104). As is known, segmentation, generally, is the process of groupingtogether into a single entity (object) pixels that have something incommon. In the present invention, the purpose of object segmentation isto group or segment out red regions (mouth or red retina). In thesegroups, features are extracted or calculated as will be described laterherein.

Object segmentation is used in the present invention to detect redpixels and regions. The normalized LAB space is proved to be effectivefor segmenting out mouth-like patterns. As is well known, a LAB colorspace is a color-opponent space with dimension L measuring lightness,and A and B representing color-opponent dimensions. FIG. 4 depicts thenormalized LAB image of FIG. 3. In this depiction the whole image istransformed into LAB space for illustration purposes. However, in theactual implementation only pixels in the bounding rectangles are neededto be transformed into LAB space. The values of L, A, and B are firstnormalized to [0, 1] and then mapped into [0, 255]. It is observed fromexperiments that in most cases, for a mouth object, values of A and(A-B) of the red pixels are normally larger than those of other pixels.Therefore, each pixel (located at coordinate (x, y)) inside thecandidate bounding rectangle can be determined as a red pixel or notaccording to its A and (A-B) values as follows,

${b\left( {x,y} \right)} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} A_{({x,y})}} > {T_{A}\mspace{14mu} {and}\mspace{14mu} \left( {A_{({x,y})} - B_{({x,y})}} \right)} > T_{A\_ B}} \\0 & {otherwise}\end{matrix} \right.$

where b(x,y)=1 denotes a red pixel at coordinate (x, y) and b(x,y)=0denotes a non-red pixel at coordinate (x, y), T_(A) and T_(A) _(—) _(B)are thresholds determined by an automatic threshold selection technique.In a preferred embodiment, Otsu's method is used. As is well known,Otsu's method automatically performs histogram shape-based imagethresholding, or, the reduction of a graylevel image to a binary image.A connected component labeling procedure is then applied to obtain oneor more red regions. The largest red region denoted as “newFeaObj” isthen chosen for feature extraction, discussed later herein. FIG. 5depicts the segmentation results for each candidate object in FIG. 3,with the largest red region being the mouth pattern.

Due to the complex illumination variations, segmentation in a singlecolor space can not always guarantee optimum results. Therefore,segmentation in different color spaces is also applied. The featuresdescribed later will be calculated in LAB space and some other colorspaces. Features extracted from different color spaces are then combinedand utilized in the decision making step 108, described later, to give amore reliable result. In one embodiment, similar segmentation is appliedin the RGB (red, green, blue) color space to identify red pixels in thebounding rectangle of each candidate object.

The present invention is not limited to, or particularly concerned withany particular segmentation method, thresholding method, or color space.Although Otsu's method is applied in the LAB and RGB color spaces in oneexemplary implementation, any type of segmentation technology, rednessmeasurement and color spaces may be utilized as long as the mouth likered pixels can be extracted.

The next step in method 100 is feature extraction (step 106). Beforedetailing this step, we will first discuss the various features that areof interest in the present invention.

Feature k_(w)

First, we consider a horizontal scenario where the left eye and righteye are horizontally aligned.

FIG. 6 depicts a typical example of an eye pattern and its correspondingbounding box (large, exterior rectangle in the figure) output by an eyeclassifier. FIG. 6 (a) demonstrates a perfect detection scenario and (b)demonstrates a partial detection scenario, which is due to theimperfectness of the eye classifier. It has been observed that althoughsome partial detection exists in many eye classification methods, theoutput bounding box of an eye candidate is usually much larger than redretina region, which is bounded by the smaller, interior rectangle inthe figure.

FIG. 7 depicts a typical example of a mouth pattern and itscorresponding bounding box (exterior rectangle in the figure) output byan eye classifier. It has been observed that the red region (bounded bythe smaller, interior rectangle in the figure) occupies a larger portionof the object bounding box in both perfect (a) and partial (b) detectioncases.

Therefore, the first feature k_(w) is defined as follows:

k _(w) =W _(newFeaObj) /W _(EC)

, where W_(newFeaObj) denotes the width of the red region, which wassegmented out of the image in step 104 (FIG. 1) discussed above, andW_(EC) denotes the width of the bounding box output by an eyeclassifier. In most cases, the k_(w) value of a mouth pattern is largerthan that of an eye pattern.

Feature k_(h)

Feature k_(w) is effective for horizontal eye/mouth classification asshown in FIGS. 6 and 7. For vertical cases where the two eyes of oneperson are vertically aligned, feature k_(h) is proposed as follows:

k _(h) =H _(newFeaObj) /H _(EC)

, where H_(newFeaObj) denotes the height of the red region, which wassegmented out of the image in step 104 (FIG. 1) discussed above, andH_(EC) denotes the height of the bounding box output by an eyeclassifier. Similarly, in most cases, the k_(h) value of a verticalmouth pattern is larger than that of a vertical eye pattern.

Feature s

Feature s is defined as the ratio between W_(newFeaObj) andH_(newFeaObj) as follows,

s=W _(newFeaObj) /H _(newFeaObj)

Feature v

Feature v is defined as the decision score output by an eye classifier.It can be similarity score, confidence level or vote. In one embodiment,v denotes the vote of a boosting based classifier. The present inventionis not limited to or concerned with any particular eye classifier.However, AdaBoost is a good, commonly known and commonly used methodthat constructs a classifier in an incremental fashion by adding simpleclassifiers to a pool and using their weighted vote to determine theirfinal classification.

Feature m

Feature m captures the geometric relationship between objects in a givenimage. FIG. 8 shows three cases of feature m, assuming a horizontalmouth (as shown in FIG. 7) is considered. In FIG. 8, “object” representsthe candidate object (to be determined as a mouth or not) and the othercircles represent other objects in the given image. In case (a), if 1 or2 objects are found in the top side region within a predetermineddistance of the candidate object, feature m of the candidate object isflagged as “Mouth_Eye_Pair”. In case (b), m of the candidate object isflagged as ‘Eye_Eye_Pair” if there is one object found in its left orright side region within a predetermined distance. In other cases, m isflagged as “No_Pair”. FIG. 3 (c) illustrates an example where m isflagged as “No_Pair”. The “predetermined distances” referenced abovemust be determined empirically for a particular device, for example bytaking test photos of faces with a device, subjecting them to an eyeclassifier and determining typical distances between candidate red-eyeobjects that correspond to the actual eyes and mouth of the test photos.

For vertical cases, feature m can be determined in a similar way.

Now that we have outlined the features of interest in the presentinvention, we will discuss how these features are used in the decisionmaking step, which is step 108 in FIG. 1.

To determine if a candidate object is a mouth pattern or not, thepresent invention employs a decision making mechanism using the obtained8 features k^(LAB) _(w), k^(LAB) _(h), s^(LAB), k^(RGB) _(w), k^(RGB)_(h), s^(RGB), v and m.

Each candidate object's features are extracted as described above, andfor a particular candidate, if its feature k^(LAB) _(w)>k^(LAB) _(h) ork^(RGB) _(w)>k^(RGB) _(h) and at the same time k^(LAB) _(w) is largerthan a threshold t_(kw), a horizontal mouth scenario is assumed for thatparticular candidate and m is determined based on the horizontal case asdescribed above.

The decision as to whether or not the particular candidate object is amouth can then be made by examining if all other features are satisfiedwith some pre-defined thresholds in a horizontal mouth scenario. In oneembodiment, the decision logic can be as follows: the candidate objectis determined as a mouth pattern if m is not flagged as “Eye_Eye_Pair”and s^(LAB) is larger than a threshold t_(s), where tk_(kw)=0.7 andt_(s)=0.79.

For each candidate object, if its feature k^(LAB) _(h)>k^(LAB) _(w) ork^(RGB) _(h)>k^(RGB) _(w) and at the same time k^(LAB) _(h) is largerthan a threshold t_(kh), a vertical mouth scenario is assumed and m iscalculated based on the vertical case. In a similar way, the decision asto whether or not the candidate is a mouth can then be made by examiningif all other features are satisfied with some pre-defined thresholds ina vertical mouth scenario. In one embodiment, the decision logic can beas follows: the candidate object is determined as a mouth pattern if vis larger than a threshold t_(v) and Max(W^(LAB) _(newFeaObj),H^(LAB)_(newFeaObj)) is larger than a threshold t_(hw), where t_(kh)=0.6,t_(v)=6, t_(hw)=6, and Max(W^(LAB) _(newFeaObj),H^(LAB) _(newFeaObj))represents the larger value of W^(LAB) _(newFeaObj) and H^(LAB)_(newFeaObj).

If a red-eye candidate is determined to be a mouth pattern then it isexcluded from further processing, i.e. excluded from red-eye correction.Thus, the present invention enhances red-eye correction mechanismcurrently used by identifying and excluding certain “false positives,”particularly mouth patterns from red-eye correction. This enhances theoverall result and provides an improved corrected image to the user.

While the invention has been described in conjunction with severalspecific embodiments, it is evident to those skilled in the art thatmany further alternatives, modifications and variations will be apparentin light of the foregoing description. Thus, the invention describedherein is intended to embrace all such alternatives, modifications,applications and variations as may fall within the spirit and scope ofthe appended claims.

1. A method for determining if a candidate for red-eye removalprocessing is a mouth, comprising: receiving from an eye classifier aplurality of candidate red-eye objects, each of the candidate red-eyeobjects contained within a bounding rectangle; for each particularcandidate red-eye object: segmenting the candidate red-eye object toobtain a segmented red region within the candidate red-eye object'sbounding rectangle; extracting a feature k_(w), defined as the ratiobetween a width of the segmented red region and a width of the boundingrectangle; extracting a feature k_(h), defined as the ratio between aheight of the segmented red region and a height of the boundingrectangle; extracting a feature s, defined as the ratio between thewidth and height of the segmented red region; extracting a feature v,defined as a decision score from the eye classifier; extracting afeature m, defined as a geometrical relationship between the particularcandidate red-eye object and others of the plurality of candidatered-eye objects; and using the features k_(w), k_(h), s, v, and m todetermine if the particular candidate red-eye object is a mouth.
 2. Themethod as recited in claim 1, wherein extracting the feature m comprisesdetermining the geometrical relationship between the particularcandidate red-eye object and others of the plurality of candidatered-eye objects and flagging the feature m as Mouth_Eye_Pair if one ortwo of the other of the plurality of candidate red-eye objects are foundin the top side region within a predetermined distance of the particularcandidate red-eye object; flagging the feature m as Eye_Eye_Pair if oneof the other of the plurality of candidate red-eye objects is found in aleft or right side region within a predetermined distance of theparticular candidate red-eye object; and flagging the feature m asNo_Pair in all other cases.
 3. The method as recited in claim 2, whereinusing the features k_(w), k_(h), s, v, and m to determine if theparticular candidate red-eye object is a mouth includes: comparing k_(w)to k_(h), and comparing k_(w) to a threshold t_(kw) and if k_(w)>k_(h)and k_(w)>t_(kw), then a horizontal mouth condition is decided for theparticular candidate red-eye object and feature m is extracted based ona horizontal mouth condition.
 4. The method as recited in claim 3 wherem is extracted based on a horizontal mouth condition and wherein, theparticular candidate red-eye object is determined to be a mouth if m isnot flagged as Eye_Eye_Pair and feature s is larger than a thresholdt_(s), where t_(kw)=0.7 and t_(s)=0.79.
 5. The method as recited inclaim 2, wherein using the features k_(w), k_(h), s, v, and m todetermine if the particular candidate red-eye object is a mouthincludes: comparing k_(h) to k_(w), and comparing k_(h) to a thresholdt_(kh) and if k_(h)>k_(w) and k_(h)>t_(kh), then a vertical condition isdecided for the particular candidate red-eye object and feature m isextracted based on a vertical mouth condition.
 6. The method as recitedin claim 5 where m is extracted based on a vertical mouth condition andwherein, the particular candidate red-eye object is determined to be amouth if v is larger than a threshold t_(v) and Max(W^(LAB)_(newFeaObj),H^(LAB) _(newFeaObj)) is larger than a threshold t_(hw),where t_(kh)=0.6, t_(v)=6, t_(hw)=6, and Max(W^(LAB) _(newFeaObj),H^(LAB) _(newFeaObj)) represents the larger value of W^(LAB)_(newFeaObj) and H^(LAB) _(newFeaObj).
 7. The method of claim 1, whereinsegmenting the candidate red-eye comprises: for pixels in the boundingrectangle received from the eye classifier, transforming the pixels fromRGB color space to LAB color space; calculating thresholds for A and(A-B), denoted as T_(A) and T_(A) _(—) _(B) respectively, using anautomatic threshold selection algorithm; applying a binarizationprocedure by determining if each pixel inside the bounding rectanglereceived from the eye classifier is a red pixel or non-red pixelaccording to the following equation:${b\left( {x,y} \right)} = \left\{ \begin{matrix}1 & {{{if}\mspace{14mu} A_{({x,y})}} > {T_{A}\mspace{14mu} {and}\mspace{14mu} \left( {A_{({x,y})} - B_{({x,y})}} \right)} > T_{A\_ B}} \\0 & {otherwise}\end{matrix} \right.$ where b(x,y)=1 denotes a red pixel at coordinate(x, y) and b(x,y)=0 denotes a non-red pixel at coordinate (x, y), thebinarization procedure resulting in a binary image.
 8. The method ofclaim 7, further comprising applying a connected component labelingtechnique to the binary image to obtain one or more connected redregions, and labeling a red region with the largest size newFeaObj. 9.The method of claim 8, further comprising calculating the width WnewFeaObj and height H_(newFeaObj) of the red region newFeaObj.
 10. Themethod of claim 7, wherein transforming the pixels from RGB color spaceto LAB color space comprises: first converting the pixels from RGB colorspace to LAB color space; then normalizing the pixels in LAB space to[0, 1], and then mapping the normalized pixels into [0, 255].
 11. One ormore computer-readable media having computer-readable instructionsthereon, which, when executed by a processor, implement a method fordetermining if a candidate for red-eye removal processing is a mouth,comprising: receiving from an eye classifier a plurality of candidatered-eye objects, each of the candidate red-eye objects contained withina bounding rectangle; for each particular candidate red-eye object:segmenting the candidate red-eye object to obtain a segmented red regionwithin the candidate red-eye object's bounding rectangle; extracting afeature k_(w), defined as the ratio between a width of the segmented redregion and a width of the bounding rectangle; extracting a featurek_(h), defined as the ratio between a height of the segmented red regionand a height of the bounding rectangle; extracting a feature s, definedas the ratio between the width and height of the segmented red region;extracting a feature v, defined as a decision score from the eyeclassifier; extracting a feature m, defined as a geometricalrelationship between the particular candidate red-eye object and othersof the plurality of candidate red-eye objects; and using the featuresk_(w), k_(h), s, v, and m to determine if the particular candidatered-eye object is a mouth.
 12. The one or more computer-readable mediaas recited in claim 11, wherein extracting the feature m comprisesdetermining the geometrical relationship between the particularcandidate red-eye object and others of the plurality of candidatered-eye objects and flagging the feature m as Mouth_Eye_Pair if one ortwo of the other of the plurality of candidate red-eye objects are foundin the top side region within a predetermined distance of the particularcandidate red-eye object; flagging the feature m as Eye_Eye_Pair if oneof the other of the plurality of candidate red-eye objects is found in aleft or right side region within a predetermined distance of theparticular candidate red-eye object; and flagging the feature m asNo_Pair in all other cases.
 13. The one or more computer-readable mediaas recited in claim 12, wherein using the features k_(w), k_(h), s, v,and m to determine if the particular candidate red-eye object is a mouthincludes: comparing k_(w) to k_(h), and comparing k_(w) to a thresholdt_(hw) and if k_(w)>k_(h) and k_(w)>t_(hw), then a horizontal mouthcondition is decided for the particular candidate red-eye object andfeature m is extracted based on a horizontal mouth condition.
 14. Theone or more computer-readable media as recited in claim 13 where m isextracted based on a horizontal mouth condition and wherein, theparticular candidate red-eye object is determined to be a mouth if m isnot flagged as Eye_Eye_Pair and feature s is larger than a thresholdt_(s), where t_(hw)=0.7 and t_(s)=0.79.
 15. The one or morecomputer-readable media as recited in claim 12, wherein using thefeatures k_(w), k_(h), s, v, and m to determine if the particularcandidate red-eye object is a mouth includes: comparing k_(h) to k_(w),and comparing k_(h) to a threshold t_(kh) and if k_(h)>k_(w) andk_(h)>t_(kh), then a vertical condition is decided for the particularcandidate red-eye object and feature m is extracted based on a verticalmouth condition.
 16. An image capture device for determining if acandidate for red-eye removal processing is a mouth, comprising: aprocessor that: receives from an eye classifier a plurality of candidatered-eye objects, each of the candidate red-eye objects contained withina bounding rectangle; for each particular candidate red-eye object:segments the candidate red-eye object to obtain a segmented red regionwithin the candidate red-eye object's bounding rectangle; extracts afeature k_(w), defined as the ratio between a width of the segmented redregion and a width of the bounding rectangle; extracts a feature k_(h),defined as the ratio between a height of the segmented red region and aheight of the bounding rectangle; extracts a feature s, defined as theratio between the width and height of the segmented red region; extractsa feature v, defined as a decision score from the eye classifier;extracts a feature m, defined as a geometrical relationship between theparticular candidate red-eye object and others of the plurality ofcandidate red-eye objects; and uses the features k_(w), k_(h), s, v, andm to determine if the particular candidate red-eye object is a mouth.17. The image capture device as recited in claim 16, wherein extractingthe feature m comprises determining the geometrical relationship betweenthe particular candidate red-eye object and others of the plurality ofcandidate red-eye objects and flagging the feature m as Mouth_Eye_Pairif one or two of the other of the plurality of candidate red-eye objectsare found in the top side region within a predetermined distance of theparticular candidate red-eye object; flagging the feature m asEye_Eye_Pair if one of the other of the plurality of candidate red-eyeobjects is found in a left or right side region within a predetermineddistance of the particular candidate red-eye object; and flagging thefeature m as No_Pair in all other cases.
 18. The image capture device asrecited in claim 17, wherein using the features k_(w), k_(h), s, v, andm to determine if the particular candidate red-eye object is a mouthincludes: comparing k_(w) to k_(h), and comparing k_(w) to a thresholdt_(hw) and if k_(w)>k_(h) and k_(w)>t_(hw), then a horizontal mouthcondition is decided for the particular candidate red-eye object andfeature m is extracted based on a horizontal mouth condition.
 19. Theimage capture device as recited in claim 18 where m is extracted basedon a horizontal mouth condition and wherein, the particular candidatered-eye object is determined to be a mouth if m is not flagged asEye_Eye_Pair and feature s is larger than a threshold t_(s), wheret_(hw)=0.7 and t_(s)=0.79.
 20. The image capture device as recited inclaim 17, wherein using the features k_(w), k_(h), s, v, and m todetermine if the particular candidate red-eye object is a mouthincludes: comparing k_(h) to k_(w), and comparing k_(h) to a thresholdt_(kh) and if k_(h)>k_(w) and k_(h)>t_(kh), then a vertical condition isdecided for the particular candidate red-eye object and feature m isextracted based on a vertical mouth condition.